Stochasticity and evolutionary stability 
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In stochastic dynamical systems, different concepts of stability can be obtained in different limits. A particu- 
larly interesting example is evolutionary game theory, which is traditionally based on infinite populations, where 
strict Nash equilibria correspond to stable fixed points that are always evolutionarily stable. However, in finite 
populations stochastic effects can drive the system away from strict Nash equilibria, which gives rise to a new 
concept for evolutionary stability. The conventional and the new stability concepts may apparently contradict 
each other leading to conflicting predictions in large yet finite populations. We show that the two concepts can 
be derived from the frequency dependent Moran process in different limits. Our results help to determine the 
appropriate stability concept in large finite populations. The general validity of our findings is demonstrated 
showing that the same results are valid employing vastly different co-evolutionary processes. 

PACS numbers: 87.23.-n, 89.65.-s 05.45.-a, 02.50.Ey 



I. INTRODUCTION 

Evolutionary game theory Ql provides a powerful and uni- 
fying mathematical framework widely used in scientific ar- 
eas as diverse as biology 0|, economics (sj, and social sci- 
ences. More recently, it has been attracting growing interest 
in physics 01510], as several sophisticated techniques devel- 
oped in the physics of complex systems have provided use- 
ful insights into this interdisciplinary framework l|7J. Origi- 
nally, evolutionary game theory was formulated in terms of 
infinite populations and the corresponding replicator dynam- 
ics 1 8, 9]. As a nonlinear dynamical system, these equations 
are of great interest being formally equivalent to the well stud- 
ied Lotka-Volterra equations |01 • Much of our present intu- 
ition is based upon this deterministic framework that analyzes 
nonlinear systems of ordinary differential equations. How- 
ever, any real population has finite size and also individual- 
based computer simulations in unstructured or structured pop- 
ulations llOl m d El d m always deal with finite 
populations. As is well known in physics, the behavior of a 
finite system can depart significantly from its infinite counter- 
part. 

Recently, the concept of evolutionary stability has been 
specifically investigated for finite populations iIitI lisll and 
the traditional concept of evolutionary stability has been chal- 
lenged. Analyzing the connection between the two frame- 
works of evolutionary stability we show that the difference 
between them is not solely based on finite size effects. 

When selection is frequency independent, the idea of evo- 
lutionary stability is simple: A population evolves until a sta- 
ble fixed point is reached at which fitness is maximized. In 
contrast with this situation, in evolutionary game theory the 
fitness of an individual depends on the type and frequency of 
its competitors. Hence, the optimization of individual fitness 
can even lead to the decline of the average fitness, as in the 
Prisoner's Dilemma where each individual is better off not co- 
operating, although mutual cooperation leads to higher fitness 

The standard definition of evolutionary stability formulated 



for infinite populations Ql is equivalent to strong stability of 
the corresponding fixed point in the replicator dynamics iP): 
A strategy is an evolutionarily stable strategy (ESS) if indi- 
viduals with this strategy are always better off than a small 
fraction of mutants. A similar result holds for infinite popu- 
lations subject to external noise; In the stochastic replicator 
dynamics of Fudenberg and Harris li20l . the population stays 
nearly all the time close to an ESS if the ESS corresponds to 
an interior state or if the payoff matrix satisfies a certain def- 
initeness condition ll2lll . Moreover, in the stochastic replica- 
tor dynamics, strict Nash equilibria are always asymptotically 
stochastically stable, provided the impact of the external noise 
is not too strong. However, for finite populations the con- 
cept of evolutionary stability has been challenged, as an ESS 
defined as above is not necessarily stable anymore whenever 
the population size is finite iHtIi . Nowak et al. introduced a 
concept for evolutionary stability in finite populations (ESSn) 
proposing the additional requirement that selection has to op- 
pose replacement by other strategies due to random drift IITIi . 
For coordination games corresponding to bistable situations, 
they also derived the condition under which selection favors 
the replacement of one strict Nash equilibrium by the other 
They found that replacement occurs if the unstable fixed point 
that is always present in coordination games is closer than 
1/3 to the strategy to be replaced. This finding obtained for 
weak selection contradicts the idea of evolutionary stability 
in infinite populations, where such a replacement cannot oc- 
cur Furthermore, even in simulations of finite populations 
such a replacement has not been demonstrated yet, as selec- 
tion is usually strong in these systems. Here we show that 
these two apparently contradicting concepts actually emerge 
as different limiting results of a unified treatment. Both the 
traditional ESS concept and the ESSn concept can be derived 
from the frequency dependent Moran process in different lim- 
its. However, these results do not rely on Moran dynamics. 
Indeed, their general validity is shown to apply to other types 
of co-evolutionary dynamics^such as the frequency depen- 
dent Wright-Fisher process I22ll and the local update process 
investigated in ll23ll . 
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The remainder of this paper is organized as follows: In 
Sec. in] we introduce the frequency dependent Moran process 
and discuss how different limits lead to different concepts of 
evolutionary stability. In Sees. IIIII and II VI we show how the 
results transfer to the Local update process and the frequency 
dependent Wright-Fisher process. Finally, we discuss impli- 
cations on the fixation times in Sec.fVl 



II. THE FREQUENCY DEPENDENT MORAN PROCESS 

In evolutionary game theory, the fitness of an individual is 
determined by the payoff from its interactions with others. An 
A individual will obtain a payoff a from an interaction with A 
individuals and b from B individuals. Similarly, B individuals 
gain c from A and d from B. Hence, the payoffs are given by 
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for the types A and B, respectively. N is the population size 
and i is the number of A individuals in the population. Self- 
interactions have been excluded. For the frequency depen- 
dent Moran process the fitness of an individual is a lin- 
ear combination of a background fitness and the payoff from 
interactions, = 1 — w + wtta for A individuals and anal- 
ogously for B. The parameter < w < 1 measures the in- 
tensity of selection. For w ^ 1, selection is weak and the 
payoff of the game has only a marginal influence, whereas the 
background fitness becomes negligible for w ^ 1. 

We assume that the fitness is always positive. An individ- 
ual chosen proportional to its fitness produces one identical 
offspring which replaces a randomly chosen individual. The 
probability to increase the number of A individuals from i to 
I + 1 is 

ifA N^i 



(3) 
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whereas the probability to decrease it from i to i 

ifA + iN -i)fBN- 

Since (N) = and r+(0) = 0, this process has absorbing 
states at i = and i — N. For large TV, the Master equa- 
tion describing this process can be approximated by a Fokker- 
Planck equation with drift a{x) w T'^{x) — T~{x) and dif- 
fusion b'^lx) w {T+{x) + r-(.T))/iVjWhere x^i/N\^ the 
fraction of type A in the population 12311 . This Fokker- Planck 
equation corresponds to the stochastic differential equation 

dx = a{x)dt + b{x)dW{t), (5) 

where W{i) is the Wiener process, {W{s)W{t)) — min(s,t) 
1I24I1 . For the frequency dependent Moran process, we have 



a{x) ~ x{\ — x) 



xfAix) + (1 - x)fB{x) 



(6) 



,2/ N _ X{l-X) fA{x)+fB{x) 

^ N xfA{x) + {l-x)fB{x)- ^ ' 



For fixed selection intensity w and N ^ 00, b{x) vanishes 
and a deterministic replicator equation is obtained from the 
frequency dependent Moran process ['23]. In this determinis- 
tic limit, a stable coexistence of A and B is possible. 

However, if Nw ^ 1, the stochasticity is retained even in 
the limit of infinite population size and the system will eventu- 
ally get absorbed in the state a; = or x = 1. The probability 
that the system gets absorbed in a; = 1 starting from .tq (in 
other words, the probability to reach the state with A individ- 
uals only) can be computed from the drift and diffusion term 
as 
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where S{x) 



'S^n^)d^-dy, (8) 



and r(z) = 2a{z)/b'^{z) (2J. It is noteworthy that there 
are similarities between our absorption probabilities Eq. (|8|i 
and the results of Ref. in which a quantum description of 
game dynamics has been explored. 

For neutral selection, w = {), the drift term vanishes, 
a{z) = 0, and the fixation probability is simply the initial 
fraction of type A, (f>{xo) = xq. For w <C 1, the ratio r(z) 
that determines the fixation properties of the process becomes 



r(z) « Nw{tta - ttb) = Nw (oLZ + /?) , 



(9) 



where a = (a-6-c + d)N/{N - 1) and /3 = (-a + bN - 
dN + d)/ {N — 1). A particular interesting case is Nw <^ 1, 
in which the fixation probability Eq. (|8ji is given by 



(j)(x) MX+ -Nx(l - x) (a(l + x) + 3/3) . 
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For X = 1/N, this reduces to the fixation probability of a sin- 
gle mutant in the limit of weak selection, which has been com- 
puted directly from the Moran process in ITtIi . Selection fa- 
vors A replacing B if the fixation probability of a single A mu- 
tant is higher than the fixation probability of a neutral mutant, 
which amounts to (f>{x) > x. For the evolutionary stability of 
the state with B individuals only, the condition (/)(a;) > a; is 
of interest in the vicinity of x = 0. After derivation with re- 
spect to X we find for a; = the condition 5(1) < 5"(0) = 1. 
For — > 00 and w ^ with Nw <C 1, the development of 
5(1) < 1 reduces to a + 26 > c + 2d. 

For a > c and b > d, the fitness of A is larger than the 
fitness of B for any number of mutants. In this case, the con- 
dition 5(1) < 1 is always fulfilled and A will invade the pop- 
ulation with a higher probability than a neutral mutant. On the 
other hand, for a < c and b < d, the strategy A is disadvanta- 
geous and the probability for its invasion is smaller than that 
of a neutral mutant. However, the most interesting case is as- 
sociated with those situations in which tta — ttb changes sign 
for a certain x. Here, we concentrate on coordination games 
in which a > c and b < d. In these games, the situation 
with 100% A or 100% B individuals are strict Nash equilib- 
ria: If the population is dominated by type A, a B mutant is 
disadvantageous, but if B dominates, A individuals become 
disadvantageous. Coordination games are bistable, i.e. for an 
intermediate number of mutants the preferred direction of the 
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evolutionary process changes, as the drift term a{x) changes 
sign. Whereas in the deterministic replicator equation x = 
and x = 1 are stable fixed points, stochasticity can lead from 
one point to the other, similar to a physical process in which 
stochastic fluctuations enable a particle to overcome an en- 
ergy barrier. In asymmetric situations, stochasticity favors 
one equilibrium over the other, depending on the position of 
the unstable fixed point x* between them. When this point is 
close to 0, it is relatively easy for A to invade, as a small group 
of A individuals changes the sign of the drift term a{x). More 
specifically, for a > c and b < d selection favors A replacing 
Bif 



d-b 1 
d 3 



-b- 



(11) 



where x* is the unstable equilibrium of the game at which the 
fitness of both strategies is the same, it a = ttb- This result 
coined as 1/3-rule by Nowak et al. can be derived directly 
from the exact fixation probabilities of the Markov chain and 
is valid for Nw ^ 1. It is a central result of the ESSn con- 
cept flTll . It can also be obtained in the same limit from Eq. 
ilO\ with X = 1/N. Furthermore, Eq. jlOt shows that this 
1/3-rule extends beyond the case of a single mutant and is 
also obtained if the fixation probability of a fixed number of 
mutants is investigated. 

In general, the stochasticity arising from finite populations 
is qualitatively different from the Gaussian noise incorporated 
in Eq. (|5|i. Therefore, it is remarkable that the approximation 
of population dynamics in finite populations with stochastic 
differential equations agree for weak selection perfectly with 
the direct calculation for arbitrary N, although the Langevin 
approximation implicitly assumes large populations. How- 
ever, if w remains bounded away from zero, we can no longer 
assume Nw <C 1 for N ^ oo and the 1 /3-rule is violated, as 
illustrated in Fig.Q] In this case, we obtain for w <C 1 
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[erf (zi) - erf (zq)] , 



is the error function and 



where erf {x) — dy e 

Zi = {ai + /3)y^ Nw/{2a). For coordination games with 
a > c and b < d, the right hand side of Eq. (I12t diverges 
for N ^ (X and fixed w 1. Therefore, 5(1) > 1 and 
selection never favors A replacing B in this setting. As B 
and A are ESS in coordination games, this is exactly what 
the traditional evolutionary stability predicts. Hence, we have 
shown that evolutionary stability depends on the quantity Nw, 
i.e. the size of the population times the intensity of selec- 
tion. Nw <C 1 leads to the ESSn concept from IITIi. whereas 
Nw ^ 1 reveals the traditional ESS concept 1 1]. There is no 
contradiction between these notions of evolutionary stability: 
They both constitute two extreme limits of the quantity Nw. 
Indeed, as explicitly shown in the next two Sections, this re- 
sult does not depend on the frequency dependent Moran pro- 
cess adopted so far. 



For intermediate values of Nw, the position of the unstable 
equilibrium x* such that selection favors the replacement of 
Bhy A can be solved numerically from S{1) = 1 for a given 
payoff matrix, as shown in Fig. [J For x < x* selection fa- 
vors replacement of B by A, for x > x* this replacement is 
not favored. Eq. (I12> depends on the payoffs as a function of 
a and /3 and not only as function of x*. Therefore, there is 
no universal relation analog to the 1/3-rule for fixed Nw. In 
general, the position of the unstable equilibrium will depend 
on all entries in the payoff matrix and not only on a certain 
combination of them. In Fig. [2 one parameter of the payoff 
matrix is varied and the resulting position of x* is shown. For 
coordination games (a > c and d > b) with d > a, B is 
a Pareto optimal equilibrium that maximizes payoffs, as the 
equilibrium with A players only yields a lower payoff. Fur- 
thermore, the condition a — c > d — b implies that A is risk 
dominant, i.e. equilibrium A has a larger basin of attraction 
than B. If the level of stochasticity is small, Nw ^ 1, re- 
placement of the Pareto optimal equilibrium only occurs if the 
basin of attraction of this equilibrium is very small. For high 
stochasticity, Nw <C 1, replacement of the Pareto optimal by 
the Risk dominant equilibrium is much more likely. 

We note that we do not consider a thermodynamical limit 
here, as the properties of the system are not conserved. A 
more detailed account on the thermodynamical limit of the 
frequency dependent Moran process is given in iIztIi . 



III. THE LOCAL UPDATE PROCESSES 

To demonstrate the general validity of our result for evolu- 
tionary stability, we show that the two limits Nw <^ 1 and 
Nw ^ 1 lead to the same result for two vastly different pro- 
cesses. In this Section, we consider the local update process 
discussed in ^3 ■ In this process, two individuals are chosen 
at random from the population. Only the choice of two differ- 
ent individuals can change the composition of the population. 
In mixed pairs, the A individual replaces the B individual with 
probability 
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P=2 



W -Ka- T^B 

2" 'K^ ' 



(13) 



where Att is the maximum possible payoff difference. The B 
individual replaces the A individual with probability 1 — p. As 
for the Moran process, w determines the intensity of selection: 
For small w, stochastic effects are frequent, whereas w — > 1 
leads to strong selection. This process is described by the 
transition probabilities 



i N-i 
N N 



1 _|_ W TTA 

2 2 Att 



(14) 



Again, the birth-death process can be approximated by a 
stochastic differential equation, where the drift a{x) and the 
diffusion b(x) are given by 1:2^ 

w 

a{x) sa x{l - x)-—[nA{x) - ttb{x)] (15) 
Att 



b^{x) « x{l~x)/N. 



(16) 
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FIG. 1 : (color online) The favored equilibrium in a bistable situation 
given by a coordination game depends on the position of the unstable 
equilibrium x* . Here, the position of x* for which selection in finite 
populations favors the replacement of the Pareto optimal equilibrium 
B (with the higher payoff) by the risk dominant strategy A (with a 
larger basin of attraction) is computed numerically for different A'^ 
and fixed w = 0.001. x* is shifting towards the pure B strategy with 
increasing Nw, i.e. decreasing degree of stochasticity. For Nw <C 1, 
the equilibrium has to be at x* < 1 /3 for A to replace B irrespective 
of the process at stake (the local update process, the Moran process, 
or the frequency dependent Wright-Fisher process). For Nw — » cxj 
the equilibrium converges to the pure B strategy, as expected for the 
replicator dynamics where selection never favors A replacing B if 
B is a stable fixed point. Although these limiting results hold for 
all three processes, the position of x* will depend on the specific 
co-evolutionary process and the payoff matrix (a = 1, fe = 0, c = 
3 - 2/x* < -3, d = 2, w = 0.001). 



In this case, r(a;) — 2Nw{ax + /?)/ Att , which is Unear in x 
for arbitrary intensity of selection w. Hence, we can compute 
the fixation probabiUty Eq. (|8} expUcitly, 



erf [C{x)] ~ erf [C(0)] 
erf [C(l)]-crf [C(0)] ' 



(17) 



where C.{x) = {ax + f3)y^Nw/{aAn). For weak selection 
w <^ 1, the expansion of the error functions in Eq. leads 
to Eq. dlOt again with w 2w/Att. For x = 1/N, this re- 
duces exactly to the resuh derived directly from the Markov 
chain in l23ll . Again, we can ask under which circumstances 
selection favors A replacing B. Since r(a;) is up to a constant 
factor identical to the corresponding function for the Moran 
process for weak selection, Eq. (I12> is again obtained for the 
local update process. However, here it is valid for any selec- 
tion intensity w. 

For Nw ^ 1, Eq. il H is recovered again. Hence, the 
1/3-rule is valid for weak selection even for the local update 
process. 

However, for Nw ^ 1 selection will never favor A replac- 
ing _B in a coordination game as ^(x) < x for a; -C 1, see also 



Fig-G] This implies that any finite w guarantees evolutionary 
stability in the limit N ~> oo. Hence, Nw ^ 1 leads back 
to the traditional concept of evolutionary stability as obtained 
above for the Moran process. 



IV. THE FREQUENCY DEPENDENT WRIGHT FISHER 
PROCESS 



As a third example, we consider the frequency dependent 
Wright-Fisher process, in which individuals reproduce pro- 
portional to their fitness = 1 — w + wtta, similar to the 
Moran process. However, here reproduction is not directly 
connected to death. Instead, the new generation is sampled at 
random from a large population constituted by the offspring of 
all individuals. Hence, each time step corresponds to an en- 
tire generation with N time steps in the Moran process. The 
dynamics of this process is given by the transition matrix i23l 



N 

3 )\ifA + {N-l)fB) \l.fA + {N-l).f, 



{N-i)fE 



resulting from binomial sampling. As this matrix is not tri- 
diagonal, we explicitly derive a{x) and b{x). The drift term is 
given by a{x) = {xt+At — xt)/ At, where At is the time step 
and Xt is the fraction of A individuals at time t, Xt ~ it/N. 
With At = 1/N, this yields 



a{x) « Nx{l ~ x)- 



Ia{.x) - fsix) 



(18) 



xfAix) + (1 - x)fB{x) ' 

Similarly, the diffusion b'^{x) = {{xt+^t — Xtf) / Atis 

b^{x) « x{l-x) (19) 
fAjx) feix) + Nxjl ~ x)ifAix) - fsix))^ 
{x!A{x) + {l~x)fB{x)f 

Again, we first consider weak selection. The population 
size enters in the diffusion term as Nw"^ . Hence, for weak 
selection Nw"^ <C 1 is required (in contrast to the processes 
discussed above, where Nw <^ 1 was sufficient). For weak 
selection, the diffusion term becomes h{x) = x(\ — x) 
whereas the drift term is a (x) = Nx{l — x) {fA{x) — fsix))- 
Hence, r{x) « 2Nw {tta — tt^), which is up to a factor 2 
identical to the corresponding equation for the Moran process, 
cf. Eq. Consequently, all results for weak selection, trans- 
fer to the frequency dependent Wright-Fisher process. In par- 
ticular, Eqs. ilQ\ and (II 1> are obtained again, i.e. the 1/3- rule 
is fulfilled, as shown by a different approach in l23l . 

For w I and Nw ^ 1, the argumentation can be trans- 
ferred directly from the Moran process, which leads to a vi- 
olation of the 1/3-rule again. In other words, selection never 
favors one strategy replacing the other in coordination games 
with finite w and — > cx) as illustrated in Fig. ^ Therefore, 
the traditional ESS concept is sufficient in this limit. 
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V. FIXATION TIMES 

So far, we have only considered fixation probabilities. 
However, when the conditional average fixation time T(xo) 
for fixation in x = 1 starting at xq becomes very large, fix- 
ation probabilities are of limited interest. For systems de- 
scribed by Eq. they can be computed in an elegant way 
based on the Backward Kolmogorow equation lEsll . In partic- 
ular, T{xo) is given by 



T(xo) = N t{x,xo)dx, 



(20) 



where time is measured in elementary time steps and 



t{x,x,) = ^(^-;'^f"M") e/or(.).-5(.) 0<.< 



Xo 



t{x,Xo) 



2<j>{x) 
VHx)' 



'Jo^^^^-^^Sil)- S{x)) xo<x<l. 



For neutral selection, w = 0, we have S{x) 



{x) ~ X, 



and b{x) = \/2x{l — x)/N for the Moran process. T{xn) is 
given by 



T{xo) = -iV'(l/a;o-l)ln(l-xo). 



(21) 



and large N, this reduces to T(l/N) 
N{N — 1), which is identical to the result given in ll28ll . 



For Xo 

_ For 

the local update process, we obtain T(l/A^) « 2A^(A^— l)for 
iV ^ 1. Similarly, the Wright-Fisher process yields b{x) = 
^/x{l - x), which results in the time T{l/N) « 2{N - 1). 
For w > 0, the fixation times can be computed from the inte- 
gral Eq. i2Q\ . As this can only be done numerically in general, 
two examples are given in Fig. (|2}, where the asymptotics de- 
rived in 12811 for the Moran process with li; = 1 is found for all 
three processes. Since any w < 1 can be mapped to w = 1, 
this asymptotics is of general validity llsoll . 



VI. SUMMARY 

We have shown how to naturally relate the concept of ESS n 
and ESS. In this context, the quantity Nw, i.e. the intensity of 
selection times the population size, plays the role of an or- 
der parameter which determines the concept of evolutionary 
stability under consideration. Similarly, in population genet- 
ics the product of fitness difference and population size is the 
relevant parameter i25l E9il . For Nw <C 1 we find that the 
1/3-rule discussed by Nowak et al. JItIi for the Moran pro- 
cess is generally valid and extends from the consideration of 
a single mutant to the more general case of the invasion of 
a small number of mutants. Moreover, the concept of evolu- 
tionary stability in finite populations extends to other coevo- 
lutionary processes beyond the scope of the frequency depen- 
dent Moran process. However, for fixed w and ^ 1, this 



concept is replaced by the traditional concept of evolutionary 
stability. We have demonstrated that the transition between 
these concepts is continuous, although the function that links 
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FIG. 2: (color online) For a game with mixed Nash equilibrium (pay- 
off matrix given in the figure), the conditional average fixation time 
for a single mutant T{l/N) increases exponentially with the pop- 
ulation size even for weak selection, w = 0.001 (symbols). The 
fixation times for the local update process (filled squares) and the 
Wright-Fisher process (triangles) are indistinguishable (Note, how- 
ever, that the intrinsic time-scales differ by a factor N, see text). The 
corresponding fixation times for the Moran process (circles) increase 
slower, but still exponentially. Lines: For a coordination game (pay- 
off matrix given in the figure, w = 0.1), the conditional average 
fixation time for a single mutant T(l/N) decreases slower than ex- 
ponentially for the Moran process (full line), the local update (dashed 
line), and the Wright-Fisher process (dotted line). Specifically, for 
the Moran process it has been shown that it increases as ~ TV In A*' 

Ell. 



both processes may be distinct for different processes. The 
results for the fixation times match perfectly with this picture: 
Whenever selection leads to a pure strategy, fixation is faster 
than for neutral selection ll28ll . However, in games with mixed 
Nash equilibria, the time for the fixation of a pure strategy in- 
creases exponentially with N, which is consistent with a very 
fast decay of the stationary distribution of the Moran process 
llsoll and the prediction of the replicator dynamics that the pure 
strategies are unstable fixed points. 
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